Low-Rank Approximation of Matrix Product in One Pass

We consider the following problem: given two matrices A and B, find a rank-r approximation of their product ATB. This type of linear algebra problem has many applications in the machine learning and statistics domain. For example, if A = B, then this general problem reduces to the well-known problem of finding principal components of a given data matrix. Another example is the low-rank approximation of a co-occurrence matrix ATB, where A may be a user-by-query matrix and B may be a user-by-ad matrix, so ATB computes the joint counts for each query-ad pair. As a third example, ATB can be regarded as a cross-correlation matrix between two sets of variables (e.g., two different genomic datasets). A low-rank approximation of their correlation matrix ATB can be used as a tool for understanding the association between different variables (e.g., the canonical-correlation analysis [3]).